fix ORM circular dependency #161
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
NicolasPennie
force-pushed
the
nick/orm-refactor
branch
from
794d277 to
48f0661
Compare
Can we add CI job that check that generated code is up to date? |
danenbm
reviewed
Jan 22, 2024
Comment on lines
-104
to
+116
| This script downloads these programs from mainnet and puts them in the `programs/` folder. | ||
|
|
||
| This script grabs all the code for these programs and compiles it, and chucks it into your programs folder. Go grab some coffe because this will take a while/ | ||
| If you get some permissions errors, just sudo delete the programs directory and start again. |
There was a problem hiding this comment.
This is a stale update, as the script now does in fact download the programs from mainnet and put them into programs/.
Comment on lines
-182
to
+204
| and set Solana Rust logs to error level (it is already set to error level now in the current docker compose file): | ||
|
|
||
| and set Solana Rust logs to error level: |
There was a problem hiding this comment.
Also did you mean to revert this change?
Comment on lines
+11
to
+12
| borsh = { version = "0.9.3", optional = true } | ||
| borsh-derive = { version = "0.9.3", optional = true } |
There was a problem hiding this comment.
Could these be set to workspace = true and can it be borsh = "~0.10.3" to be the same as the other packages in the workspace? Or does this package require borsh 0.9.3 for some reason?
FYI right now at the workspace level, the dependency for token metadata is mpl-token-metadata = "=2.0.0-beta.1" which requires borsh ~0.10.3. Soon I plan to switch to mpl-token-metadata Rust client which can use old or new borsh, but right now I think its missing some stuff required for blockbuster.
Comment on lines
+65
to
+69
| impl Default for RoyaltyTargetType { | ||
| fn default() -> Self { | ||
| Self::Creators | ||
| } | ||
| } |
There was a problem hiding this comment.
Do you want this to be Creators since right below default for asset sets it to Unknown?
kespinola
approved these changes
Jan 22, 2024
| @@ -0,0 +1,2 @@ | |||
| pub mod r#enum; | |||
There was a problem hiding this comment.
Possibly just have the enums defined in the model/mod.rs?
| @@ -0,0 +1,2 @@ | |||
| pub mod r#enum; | |||
There was a problem hiding this comment.
Other approach is to make these pub in the migration file that introduced them and reference by the introducing migration file when needed by other migrations. I have done this in the past. works well enough.
Yes |
It generates the code based on querying a live DB. It's do-able but feels overkill atm. |
kespinola
referenced
this pull request
in rpcpool/digital-asset-rpc-infrastructure
Jan 26, 2024
* Update raw name and raw symbol for existing NFTs (#139) Update the raw name and raw symbol from the onchain data for existing NFTs when reingesting. This allows correction of incorrect values during reprocessing on an existing index. * Upstream Helius features (#133) * Bubblegum Update Metadata Version 1 (#134) * Add code to index Bubblegum Update Metadata * Update rust toolchain file * Fix moved variable after merge * Add code from mintV1 that allows for empty URI * Ordering using asset.seq initially applied to update_metadata * Add simple check for whether asset was decompressed to Bubblegum transformers * Don't prevent sequence number update when already decompressed * Add sequence number to downloading metadata background task * Add sequence number migration (Sea ORM not regenerated yet) * Regenerate Sea-ORM types * Use new sequence numbers for Bubblegum Update Metadata * Extra condition to protect out of order creator verification * Remove base_info_seq for each creator and add creators_added_seq to asset table * Regenerate Sea-ORM types * Change creator metadata updates to use new creators_added_seq * Factor out common creator update code to helper function * Update to latest blockbuster beta * Use less than or equal for download metadata seq check * Index verified for token metadata collection * Add slot_updated to initial asset upsert, and removed duplicate items * Remove asset_was_decompressed Replaced with WHERE clauses on each upsert. Move remaining upserts from mint_v1 to db.rs. Remove upsert to asset_v1_account_attachments from mint_V1. Combine upserts for asset base info and royalty amount. * Rename royalty_amount_seq to base_info_seq * Fix typo in WHERE clause * Do not delete existing creators in mint_v1 * Update comments around database txns * Use transaction in mint_V1 and update_metadata * Use transaction for other Bubblegum instructions asset table updates * Fix tree_id key index in update_metadata * Remove use of was_decompressed flag on asset table * Add migration to remove was_decompressed and regenerate SeaORM types * Combine upsert_asset_base_info and upsert_creators and add lock * Remove unneeded creators_added_seq * Switch to EXCLUSIVE mode lock * Add NULL condition check on asset.seq * Refactored creator indexing * Use new Blockbuster that always updates all creators and verification status. * Remove deleting creators with lower sequence numbers as it would not work due to race conditions. * Add concept of "empty" creator value to support Bubblegum empty creator arrays. * Add filtering out of old creators or having no creators to DAS code. * Also get authority and tree_id accounts from Bubblegum during mint and update_metadata. * Add conditions to creator upsert, add another check at DAS API level * Rename asset_creators.verified_seq back to just regular seq * Remove unneeded condition on asset_authority upsert * Apply stale creator filtering to all DAS API queries * Use latest blockbuster beta release * Remove download_metadata_seq and add URI match check instead * Fix task URI initial query * Regenerate Sea ORM types without download_metadata_seq * asset_grouping.verified option remove * Fix filtering for getAssetsByCreator * Update to blockbuster 0.9.0-beta.5 and mpl-bubblegum 1.0.1-beta.4 * Configurable account streams (#148) * Make workers configurable Make workers fully configurab le and remove reference to the plerkle plugin. * fix lifetime --------- Co-authored-by: Kirill Fomichev <[email protected]> * fix: splt tokens with no token stanard are incorrectly categorized as single. Logic in token account updates would change owner when any token account had amount > 0 would triggers spam updates of the owner of the asset with any transfer. (#151) * Improve workspace usage (#141) * fix: remove decompress ix handling and use db transactions (#156) * Improvements for consistency NFT indexing and query consistency (#157) * Danenbm/bubblegum sequence tests 2 (#160) * Add script to forward transactions an check database results * Fix ordering and add debug info * Add remaining non-creator/non-collection tests * Require asset and cl_items files to exist * Add asset_creators and asset_grouping tests * Add verify_creator and verify_collection tests * Add more collection verification tests * Move test data to subirectory * Move repeated code to functions * Add support for running sequences in reverse * Add instructions to README for running test script * Minor README update * feat: `getSignaturesForAsset` endpoint on top of new `cl_audits_v2` table (#155) * add migration files for cl_audits_v2 * add types * ingester * add getSignaturesForAsset endpoint * refactor to resolve merge conflict related bugs * address clippy error * rename to get_asset_signatures * add instruction type update_metadata * add error log if instruction is unknown * add sort order changes --------- Co-authored-by: Nicolas Pennie <[email protected]> * fix ORM circular dependency (#161) * fix ORM circular dependency * PR comments --------- Co-authored-by: Linus Kendall <[email protected]> Co-authored-by: Tahsin Tunan <[email protected]> Co-authored-by: Michael Danenberg <[email protected]> Co-authored-by: Kirill Fomichev <[email protected]> Co-authored-by: Nicolas Pennie <[email protected]>
Draft
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
A nasty circular dependency existed between the ORM migrations and the auto-generated files. This was dangerous because there is no guarantee of alignment between the Rust types and DB schema. We only need to align on the DB schema and the rest will take care of itself now.
Changes
Testing
Running in Helius prod. I did some sanity checks by indexing 1k cNFTs and verifying that the queries worked as expected.